Systems and methods for controlling autofocus operations

ABSTRACT

A method for performing auto-focus in a camera is disclosed. The method includes: receiving, from a tracking system for tracking a position of a medical instrument, a signal; determining, based on the received signal, that the medical instrument is removed from a field of view of the camera; in response to determining that a continuous auto-focus mode for the camera is enabled: retrieving, from a database, a first focus distance value representing a focus distance that was most recently set with intent for the camera; and automatically updating a focus distance of the camera to the first focus distance value.

TECHNICAL FIELD

The present disclosure relates to medical imaging and, in particular, tooptical imaging systems suitable for use in image-guided medicalprocedures.

BACKGROUND

Digital microscopes support advanced visualization during medicalprocedures. For example, digital surgical microscopes provide magnifiedviews of anatomical structures during a surgery. Digital microscopes useoptics and digital (e.g. CCD-based) cameras to capture images inreal-time and output the images to displays for viewing by a surgeon,operator, etc.

In image-guided medical applications, such as surgery or diagnosticimaging, accurate three-dimensional (3-D) visualization of patientanatomy and surgical tools is crucial. A medical navigation system isoften used to support image-guided surgery. In an exemplary medicalnavigation system, an optical imaging system may be provided forgenerating 3-D views of a surgical site. A positioning system, such as amechanical arm, may support the optical imaging system and facilitatemaneuvering the optical imaging system to an appropriate position andorientation to maintain alignment with a viewing target.

The optical imaging system may be adapted to perform auto-focus, whichenables a camera of the optical imaging system to automatically focus ona defined viewing target, such as a tracked medical instrument.Continuous auto-focus maintains the viewing target constantly in focus.In particular, auto-focus operations dynamically focus the image on theviewing target, enabling an operator (e.g. surgeon) to control focusduring a medical procedure without having to manually adjust the focusoptics. By virtue of the auto-focus functionality, the operator mayobserve a surgical site of interest using the camera by moving theviewing target over, or in proximity of, the surgical site.

Continuous auto-focus may be disrupted, for example, by actions of theoperator or unintended changes to the viewing target. For example, ifthe viewing target leaves a field of view of the camera, an out-of-focusimage may be produced by the camera. Even a momentary loss of focus ofthe optical imaging system may lead to serious consequences in a medicalprocedure. It is desirable to mitigate the disruptive effects resultingfrom loss of continuous auto-focus during a medical procedure.

BRIEF DESCRIPTION OF DRAWINGS

Reference will now be made, by way of example, to the accompanyingdrawings which show example embodiments of the present application andin which:

FIG. 1 shows an example medical navigation system to supportimage-guided surgery;

FIG. 2 illustrates components of an example medical navigation system;

FIG. 3 is a block diagram illustrating an example control and processingsystem which may be used in the example medical navigation system ofFIGS. 1 and 2;

FIG. 4A shows the use of an example optical imaging system during amedical procedure;

FIG. 4B is a perspective view of an example embodiment of an opticalimaging system and a plurality of tracking markers;

FIG. 5 is a block diagram illustrating components of an example opticalimaging system 500;

FIG. 6 shows, in flowchart form, an example method for performingauto-focus in a camera of the optical imaging system of FIG. 5;

FIG. 7 shows, in flowchart form, another example method for performingauto-focus in a camera of the optical imaging system of FIG. 5; and

FIGS. 8A-8B illustrate the effects on focus distance of a camera whichresult from movement of a tracked viewing target.

Like reference numerals are used in the drawings to denote like elementsand features.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

In one aspect, the present disclosure describes a processor-implementedmethod for performing auto-focus in a camera. The method includes:receiving, from a tracking system for tracking a position of a medicalinstrument, a signal; determining, based on the received signal, thatthe medical instrument is removed from a field of view of the camera; inresponse to determining that a continuous auto-focus mode for the camerais enabled: retrieving, from a database, a first focus distance valuerepresenting a focus distance that was most recently set with intent forthe camera; and automatically updating a focus distance of the camera tothe first focus distance value.

In some implementations, the method may further comprise storing, in thedatabase, a predetermined number of most recent focus distance valuesfor the camera and timestamps associated with the focus distance values.

In some implementations, retrieving the first focus distance value maycomprise: obtaining, for each focus distance value stored in thedatabase, an associated speed value, the associated speed valuerepresenting an approximate speed of a tracked point of the medicalinstrument at the focus distance; retrieving, from the database, a mostrecently stored focus distance value having an associated speed that isbelow a predefined threshold speed.

In some implementations, the associated speed may be obtained based on afinite difference computation using focus distance values stored in thedatabase.

In some implementations, the predefined threshold speed may be 0.1 meterper second.

In some implementations, the camera may be configured to automaticallyfocus to a predetermined point relative to the medical instrument.

In some implementations, the continuous auto-focus mode may be activatedvia a voice input or activation of a foot pedal.

In some implementations, detecting that the medical instrument isremoved from the field of view of the camera may comprise determiningthat the predetermined point is no longer within the field of view ofthe camera.

In some implementations, detecting that the medical instrument isremoved from the field of view of the camera may comprise determiningthat an approximate speed of the medical instrument exceeds a predefinedthreshold speed.

In some implementations, the method may further comprise computing aspeed curve representing approximate speeds of the medical instrumentwhich are associated with the stored focus distance values.

In another aspect, the present disclosure describes a medical navigationsystem to support a medical procedure. The medical navigation systemincludes a tracking system for tracking a position of a medicalinstrument, a surgical camera for imaging a target surgical site, and aprocessor coupled to the tracking system and the surgical camera. Theprocessor is configured to: determine, based on a signal from thetracking system, that the medical instrument is removed from a field ofview of the surgical camera; in response to determining that acontinuous auto-focus mode for the surgical camera is enabled: retrieve,from a database, a first focus distance value representing a focusdistance that was most recently set with intent for the surgical camera;and automatically update a focus distance of the surgical camera to thefirst focus distance value.

In yet another aspect, the present disclosure describes an opticalimaging system for imaging a target during a medical procedure. Theoptical imaging system includes a movable arm, a camera mounted on themovable arm, the camera capturing images of a target surgical site, anda processor for calibrating the camera. The processor is configured to:receive, from a tracking system for tracking a position of a medicalinstrument, a signal; determine, based on the received signal, that themedical instrument is removed from a field of view of the camera; inresponse to determining that a continuous auto-focus mode for the camerais enabled: retrieve, from a database, a first focus distance valuerepresenting a focus distance that was most recently set with intent forthe camera; and automatically update a focus distance of the camera tothe first focus distance value.

Other example embodiments of the present disclosure will be apparent tothose of ordinary skill in the art from a review of the followingdetailed descriptions in conjunction with the drawings.

In the present application, the phrase “access port” is intended torefer to a cannula, a conduit, sheath, port, tube, or other structurethat is insertable into a subject, in order to provide access tointernal tissue, organs, or other biological substances. In someembodiments, an access port may directly expose internal tissue, forexample, via an opening or aperture at a distal end thereof, and/or viaan opening or aperture at an intermediate location along a lengththereof. In other embodiments, an access port may provide indirectaccess, via one or more surfaces that are transparent, or partiallytransparent, to one or more forms of energy or radiation, such as, butnot limited to, electromagnetic waves and acoustic waves.

In the present application, the term “intraoperative” is intended torefer to an action, process, method, event, or step that occurs or iscarried out during at least a portion of a medical procedure.Intraoperative, as defined herein, is not limited to surgicalprocedures, and may refer to other types of medical procedures, such asdiagnostic and therapeutic procedures.

In the present application, the term “and/or” is intended to cover allpossible combinations and sub-combinations of the listed elements,including any one of the listed elements alone, any sub-combination, orall of the elements, and without necessarily excluding additionalelements.

In the present application, the phrase “at least one of . . . or . . . ”is intended to cover any one or more of the listed elements, includingany one of the listed elements alone, any sub-combination, or all of theelements, without necessarily excluding any additional elements, andwithout necessarily requiring all of the elements.

A medical navigation system may be configured to support image-guidedmedical procedures. The medical navigation system may include a trackingsystem for tracking one or more medical instruments and an opticalimaging system. The optical imaging system includes a camera for imaginga surgical site of interest during a medical procedure. The opticalimaging system may be configured to perform auto-focusing relative to atracked tool that is used in a medical procedure. The position andorientation of a tracked tool may be determined by the tracking system,and a controller of the optical imaging system may perform auto-focusingto focus the captured image on a point defined relative to the trackedtool. By moving the tracked tool over, or in proximity to, a targetsurgical site, the operator of the medical navigation system (e.g. asurgeon) may be able to observe the surgical site without manuallycontrolling focus optics of the optical imaging system.

“Continuous” auto-focus maintains a viewing target, such as a movingobject, constantly in focus. A continuous auto-focus mode for a medicalnavigation system may allow a tracked tool (e.g. a medical instrument)to be automatically kept in focus of a camera of the optical imagingsystem. This auto-focus behavior may be disrupted, for example, when thetracked tool is removed from a field of view of the camera. If thecamera loses focus, an out-of-focus image may be produced by the camera,and the operator's ability to observe the target surgical site using thecamera may be impeded.

The present application discloses improved auto-focusing capabilities ofan optical imaging system of a medical navigation system. A controllerof the optical imaging system is configured to perform operations forrecovering continuous auto-focus for a camera of the optical imagingsystem. The controller receives a signal from the tracking systemindicating a position and/or orientation of the target tool relative tothe camera. The controller determines, based on the received signal,that the target tool is removed from a field of view of the camera. Ifcontinuous auto-focus mode is enabled for the camera, the controller isconfigured to retrieve, from a database, a first focus distance valuerepresenting a focus distance that was most recently set with intent forthe camera. The focus distance of the camera may then be automaticallyupdated to the retrieved first focus distance value.

Reference is first made to FIG. 1, which shows an example medicalnavigation system 200. The example medical navigation system 200 may beused to support image-guided surgery. As shown in FIG. 1, a surgeon 201performs surgery on a patient 202 in an operating room environment. Amedical navigation system 205 may include an equipment tower, trackingsystem, displays, and tracked instruments to assist the surgeon 201during a procedure. An operator 203 may also be present to operate,control, and provide assistance for the medical navigation system 205.

FIG. 2 shows components of an example medical navigation system 205. Thedisclosed optical imaging system may be used in the context of themedical navigation system 205. The medical navigation system 205 mayinclude one or more displays 206, 211 for displaying video images, anequipment tower 207, and a positioning system 208, such as a medicalarm, which may support an optical imaging system 500. One or more of thedisplays 206, 211 may include a touch-sensitive display for receivingtouch input. The equipment tower 207 may be mounted on a frame, such asa rack or cart, and may contain a power supply and a computer/controllerthat may execute planning software, navigation software, and/or othersoftware to manage the positioning system 208. In some embodiments, theequipment tower 207 may be a single tower configuration operating withdual displays 206, 211; however, other configurations (e.g. dual tower,single display etc.) may be possible. The equipment tower 207 may alsobe configured with a universal power supply (UPS) to provide foremergency power, in addition to a regular AC adapter power supply.

A portion of the patient's anatomy may be held in place by a holder. Forexample, as shown in FIG. 2, the patient's head and brain may be held inplace by a head holder 217. An access port 12 and associated introducer210 may be inserted into the head, to provide access to a surgical sitein the head. The optical imaging system 500 may be used to view down theaccess port 12 at a sufficient magnification to allow for enhancedvisibility. The output of the optical imaging system 500 may be receivedby one or more computers or controllers to generate a view that may bedepicted on a visual display (e.g. one or more of displays 206, 211).

In some embodiments, the medical navigation system 205 may include atracked pointer 222. The tracked pointer 222, which may include markers212 to enable tracking by a tracking camera 213, may be used to identifypoints (e.g. fiducial points) on a patient. An operator, typically anurse or the surgeon 201, may use the tracked pointer 222 to identifythe location of points on the patient 202, in order to register thelocation of selected points on the patient 202 in the medical navigationsystem 205. In some embodiments, a guided robotic system with closedloop control may be used as a proxy for human interaction. Guidance tothe robotic system may be provided by any combination of input sourcessuch as image analysis, tracking of objects in the operating room usingmarkers placed on various objects of interest, or any other suitablerobotic system guidance techniques.

Fiducial markers 212 may be connected to the introducer 210 for trackingby the tracking camera 213, which may provide positional information ofthe introducer 210 from the medical navigation system 205. In someembodiments, the fiducial markers 212 may be alternatively oradditionally attached to the access port 12. In some embodiments, thetracking camera 213 may be a 3-D infrared optical tracking stereocamera. In some other examples, the tracking camera 213 may be anelectromagnetic system (not shown), such as a field transmitter, that isconfigured to use at least one receiver coil disposed in relation to thetool(s) intended for tracking. A known profile of the electromagneticfield and known position of receiver coil(s) relative to each other maybe used to infer the location of the tracked tool(s) using the inducedsignals and their phases in each of the receiver coils.

Location data of the positioning system 208 and/or access port 12 may bedetermined by the tracking camera 213 by detection of the fiducialmarkers 212 placed on or otherwise in fixed relation (e.g. in rigidconnection) to any of the positioning system 208, the access port 12,the introducer 210, the tracked pointer 222 and/or other trackedinstruments. The fiducial marker(s) 212 may be active or passivemarkers. The displays 206, 2011 may provide an output of the computeddata of the medical navigation system 205. In some embodiments, theoutput provided by the displays 206, 211 may include at least one ofaxial, sagittal, or coronal views of patient anatomy as part of amulti-view output.

The active or passive fiducial markers 212 may be placed on tools (e.g.the access port 12 and/or the optical imaging system 500) to be tracked,to determine the location and orientation of these tools using thetracking camera 213 and medical navigation system 205. A stereo cameraof the tracking system may be configured to detect the fiducial markers212 and to capture images thereof for providing identifiable points fortracking the tools. A tracked tool may be defined by a grouping ofmarkers 212, whereby a rigid body may be defined and identified by thetracking system. This may, in turn, be used to determine the positionand/or orientation in three dimensions of a tracked tool in a virtualspace. The position and orientation of the tracked tool in 3-D may betracked in six degrees of freedom (e.g. x, y, z coordinates and pitch,yaw, roll rotations), in five degrees of freedom (e.g. x, y, zcoordinates and two degrees of free rotation), and preferably tracked inat least three degrees of freedom (e.g. tracking the position of the tipof a tool in at least x, y, z coordinates). In typical use with medicalnavigation systems, at least three markers 212 are provided on a trackedtool to define the tool in virtual space; however, it is known to beadvantageous for four or more markers 212 to be used.

Camera images capturing the markers 212 may be logged and tracked by,for example, a closed-circuit television (CCTV) camera. The markers 212may be selectable to enable, assist or facilitate in segmentation of thecaptured images. For example, infrared (IR)-reflecting markers and an IRlight source from the direction of the camera may be used. In someembodiments, the spatial position and orientation of the tracked tooland/or the actual and desired position and orientation of thepositioning system 208 may be determined by optical detection using acamera. The optical detection may be performed using an optical camera,rendering the markers 212 optically visible.

In some embodiments, the markers 212 (e.g. reflectospheres) may be usedin combination with a suitable tracking system, to determine the spatialpositioning position of the tracked tools within the operating theatre.Different tools and/or targets may be provided with respect to sets ofmarkers 212 in different configurations. Differentiation of thedifferent tools and/or targets and their corresponding virtual volumesmay be possible based on the specification configuration and/ororientation of the different sets of markers 212 relative to oneanother, enabling each such tool and/or target to have a distinctindividual identity within the medical navigation system 205. Theindividual identifiers may provide information to the medical navigationsystem 205, such as information relating to the size and/or shape of thetool within the medical navigation system 205. The identifier may alsoprovide additional information, such as the tool's central point or thetool's central axis, among other information. The virtual tool may alsobe determined from a database of tools stored in, or provided to, themedical navigation system 205. The markers 212 may be tracked relativeto a reference point, or a reference object, in the operating room, suchas the patient 202.

Various types of fiducial markers may be used. The markers 212 maycomprise the same type or a combination of at least two different types.Possible types of markers include reflective markers, radiofrequency(RF) markers, electromagnetic (EM) markers, pulsed or un-pulsedlight-emitting diode (LED) markers, glass markers, reflective adhesives,or reflective unique structures or patterns, among others. RF and EMmarkers may have specific signatures for the specific tools to whichsuch markers are attached. Reflective adhesives, structures andpatterns, glass markers, and LED markers may be detectable using opticaldetectors, while RF and EM markers may be detectable using antennas.Different marker types may be selected to suit different operatingconditions.

In some embodiments, the markers 212 may include printed or 3-D designsthat may be used for detection by an auxiliary camera, such as awide-field camera (not shown) and/or the optical imaging system 500.Printed markers may also be used as a calibration pattern, for example,to provide distance information (e.g. 3-D distance information) to anoptical detector. Printed identification markers may include designssuch as concentric circles with different ring spacing and/or differenttypes of bar codes, among other designs. In some embodiments, inaddition to or in place of using markers 212, the contours of knownobjects (e.g. the side of the access port 12) could be captured by andidentified using optical imaging devices and the tracking system.

A guide clamp 218 (or more generally a guide) for holding the accessport 12 may be provided in the medical navigation system 205. The guideclamp 218 may allow the access port 12 to be held at a fixed positionand orientation while freeing up the surgeon's hands. An articulated arm219 may be provided to hold the guide clamp 218. The articulated arm 219may have up to six degrees of freedom to position the guide clamp 218.The articulated arm 219 may be lockable to fix its position andorientation, once a desired position is achieved. The articulated arm219 may be attached or attachable to a point based on the patient headholder 217, or another suitable point (e.g. on another patient support,such as on the surgical bed), to ensure that when locked in place, theguide clamp 218 does not move relative to the patient's head.

In a surgical operating room/theatre, setup of a medical navigationsystem may be complicated; numerous pieces of equipment associated withthe surgical procedure, as well as various elements of the medicalnavigation system 205, may need to be arranged and prepared. Setup timetypically increases as more equipment is added. To assist in addressingthis, the medical navigation system 205 may include two additionalwide-field cameras to enable video overlay information. Video overlayinformation can be inserted into displayed images, such as imagesdisplayed on one or more of the displays 206, 211. The overlayinformation may illustrate the physical space where accuracy of the 3-Dtracking system (which is typically part of the medical navigationsystem 205) is greater, may illustrate the available range of motion ofthe positioning system 208 and/or the optical imaging system 500, andmay help to guide head and/or patient positioning.

The medical navigation system 205 may provide tools to the surgeon thatmay help to provide more relevant information to the surgeon, and mayassist in improving performance and accuracy of port-based surgicaloperations. Although described in the present disclosure in the contextof port-based neurosurgery (e.g. for removal of brain tumors and/or fortreatment of intracranial hemorrhages (ICH)), the medical navigationsystem 205 may also be suitable for one or more of: brain biopsy,functional/deep-brain stimulation, catheter/shunt placement (in thebrain or elsewhere), open craniotomies, and/orendonasal/skull-based/ear-nose-throat (ENT) procedures, among others.The same medical navigation system 205 may be used for carrying out anyor all of these procedures, with or without modification as appropriate.

In some embodiments, the tracking camera 213 may be part of a suitabletracking system. In some embodiments, the tracking camera 213 (and anyassociated tracking system that uses the tracking camera 213) may bereplaced with a suitable tracking system which may or may not usecamera-based tracking techniques. For example, a tracking system thatdoes not use the tracking camera 213, such as a radiofrequency trackingsystem, may be used with the medical navigation system 205.

FIG. 3 is a block diagram illustrating an example control and processingsystem 300 that may be used as part of the medical navigation system 205shown in FIG. 2 (e.g. as part of the equipment tower 207). As shown inFIG. 3, the control and processing system 300 may include one or moreprocessors 302, a memory 304, a system bus 306, one or more input/outputinterfaces 308, a communications interface 310, and storage device 312.The control and processing system 300 may interface with other externaldevices, such as a tracking system 321, data storage 342, and externaluser input and output devices 344, which may include, for example, oneor more of a display, keyboard, mouse, sensors attached to medicalequipment, foot pedal, and microphone and speaker. Data storage 342 maybe any suitable data storage device, such as a local or remote computingdevice (e.g. a computer, hard drive, digital media device, or server)having a database stored thereon. In the example shown in FIG. 3, datastorage device 342 includes identification data 350 for identifying oneor more medical instruments 360 and configuration data 352 thatassociates customized configuration parameters with one or more medicalinstruments 360. The data storage device 342 may also includepreoperative image data 354 and/or medical procedure planning data 356.Although the data storage device 342 is shown as a single device in FIG.3, it will be understood that in other embodiments, the data storagedevice 342 may be provided as multiple storage devices.

The medical instruments 360 may be identifiable by the control andprocessing unit 300. The medical instruments 360 may be connected to andcontrolled by the control and processing unit 300, or the medicalinstruments 360 may be operated or otherwise employed independent of thecontrol and processing unit 300. The tracking system 321 may be employedto track one or more medical instruments 360 and spatially register theone or more tracked medical instruments to an intraoperative referenceframe. For example, a medical instrument 360 may include trackingmarkers such as tracking spheres that may be recognizable by thetracking camera 213. In one example, the tracking camera 213 may be aninfrared (IR) tracking camera. In another example, a sheath placed overa medical instrument 360 may be connected to and controlled by thecontrol and processing unit 300.

The control and processing unit 300 may also interface with a number ofconfigurable devices, and may intraoperatively reconfigure one or moreof such devices based on configuration parameters obtained fromconfiguration data 352. Examples of devices 320, as shown in FIG. 3,include one or more external imaging devices 322, one or moreillumination devices 324, the positioning system 208, the trackingcamera 213, one or more projection devices 328, and one or more displays206, 211.

Exemplary aspects of the disclosure can be implemented via theprocessor(s) 302 and/or memory 304. For example, the functionalitiesdescribed herein can be partially implemented via hardware logic in theprocessor 302 and partially using the instructions stored in the memory304, as at least one processing module or engine 370. Example processingmodules include, but are not limited to, a user interface engine 372, atracking module 374, a motor controller 376, an image processing engine378, an image registration engine 380, a procedure planning engine 382,a navigation engine 384, and a context analysis module 386. While theexample processing modules are shown separately in FIG. 3, in someembodiments, the processing modules 370 may be stored in the memory 304and the processing modules 370 may be collectively referred to asprocessing modules 370. In some embodiments, two or more modules 370 maybe used together to perform a function. Although depicted as separatemodules 370, the modules 370 may be embodied as a unified set ofcomputer-readable instructions (e.g. stored in the memory 304) ratherthan distinct sets of instructions.

FIG. 4A illustrates use of an example optical imaging system 500,described further below, in a medical procedure. Although FIG. 4A showsthe optical imaging system 500 being used in the context of a navigationsystem environment 200 (e.g. using a medical navigation system asdescribed above), the optical imaging system 500 may also be usedoutside of a navigation system environment.

An operator, typically a surgeon 201, may use the optical imaging system500 to observe a surgical site (e.g. to look down an access port). Theoptical imaging system 500 may be attached to a positioning system 208,such as a controllable and adjustable robotic arm. The position andorientation of the positioning system 208, imaging system 500, and/oraccess port may be tracked using a tracking system, such as describedabove for the medical navigation system 205. The distance between theoptical imaging system 500 (more specifically, the aperture of theoptical imaging system 500) and the viewing target may be referred to asthe working distance. The optical imaging system 500 may be designed tobe used in a predefined range of working distance (e.g. in the range ofbetween 15 and 75 centimeters). It should be noted that, if the opticalimaging system 500 is mounted on the positioning system 208, the actualavailable range of working distance may be dependent on both the workingdistance of the optical imaging system 500 as well as the workspace andkinematics of the positioning system 208. In some embodiments, theoptical imaging system 500 may include a manual release button that,when actuated, enables the optical imaging system to be positionedmanually. For example, the controller of the optical imaging system 500may be responsive to manual control input received via a user interface.

Reference is made to FIG. 5, which illustrates components of an exampleoptical imaging system 500. The optical imaging system 500 includes anoptical assembly 505 (which may also be referred to as an opticaltrain). The optical assembly 505 includes optics, e.g. lenses, opticalfibers, etc., for focusing and zooming on a viewing target.Specifically, the optical assembly 505 includes zoom optics 510 (whichmay include one or more zoom lenses) and focus optics 515 (which mayinclude one or more focus lenses). Each of the zoom optics 510 and thefocus optics 515 may be independently movable within the opticalassembly 505 for respectively adjusting the zoom and focus. Where thezoom optics 510 and/or the focus optics 515 include more than one lens,each individual lens may be independently movable. The optical assembly505 may comprise an aperture (not shown) which is adjustable.

The optical imaging system 500 may also include a zoom actuator 520 anda focus actuator 525 for respectively positioning the zoom optics 510and the focus optics 515. The zoom actuator 520 and/or the focusactuator 525 may comprise electric motors or other types of actuators,such as pneumatic actuators, hydraulic actuators, shape-changingmaterials, e.g., piezoelectric materials or other smart materials, orengines, among other possibilities. Although the zoom actuator 520 andthe focus actuator 525 are shown outside of the optical assembly 505, insome embodiments, the zoom actuator 520 and the focus actuator 525 arecomponents of, or are integrated with, the optical assembly 505. Thezoom actuator 520 and the focus actuator 525 may operate independently,to respectively control positioning of the zoom optics 510 and the focusoptics 515. The lens(es) of the zoom optics 510 and/or the focus optics515 may each be mounted on a linear stage, e.g. a motion system thatrestricts an object to move in a single axis. which may include a linearguide and an actuator, or a conveyor system such as a conveyor beltmechanism that is respectively moved by the zoom actuator 520 and/or thefocus actuator 525 to control positioning of the zoom optics 510 and/orthe focus optics 515. In some embodiments, the zoom optics 510 may bemounted on a linear stage that is driven, via a belt drive, by the zoomactuator 520, while the focus optics 515 may be geared to the focusactuator 525. The independent operation of the zoom actuator 520 and thefocus actuator 525 may enable the zoom and focus to be adjustedindependently. Thus, when an image is in focus, the zoom may be adjustedwithout requiring further adjustments to the focus optics 515 to producea focused image.

Operation of the zoom actuator 520 and the focus actuator 525 may becontrolled by a controller 530 (e.g. a microprocessor) of the opticalimaging system 500. The controller 530 may receive control input from anexternal system. such as an external processor or an input device. Thecontrol input may indicate a desired zoom/focus, and the controller 530may. in response, cause the zoom actuator 520 or the focus actuator 525to move the zoom optics 510 or the focus optics 515 accordingly, toachieve the desired zoom/focus. In some embodiments. the zoom optics 510and/or the focus optics 515 may be moved or actuated without the use ofthe zoom actuator 520 and/or the focus actuator 525. For example, thefocus optics 515 may use electrically-tunable lenses or other deformablematerial that is directly controlled by the controller 530.

The optical imaging system 500 may enable an operator (e.g. a surgeon)of the medical navigation system 205 to control zoom or focus during amedical procedure without having to manually adjust the zoom optics 510or focus optics 515. For example, the operator may provide control inputto the controller 530 verbally. e.g. via a voice recognition inputsystem, by instructing an assistant to enter control input into anexternal input device, via a user interface provided by a workstation,using a foot pedal. or by other such means. In some embodiments, thecontroller 530 may execute preset instructions to maintain the zoomand/or focus at preset values, for example, to perform auto-focusing,without requiring further control input during a medical procedure.

An external processor (e.g. a processor of a workstation or the medicalnavigation system 205) in communication with the controller 530 may beused to provide control input to the controller 530. For example, theexternal processor may provide a graphical user interface for receivinginput instructions to control zoom and/or focus of the optical imagingsystem 500. The controller 530 may alternatively or additionally be incommunication with an external input system, e.g., a voice-recognitioninput system or a foot pedal. The optical assembly 505 includes at leastone auxiliary optic 540. e.g., an adjustable aperture, which is staticor dynamic. Where the auxiliary optics 540 is dynamic, the auxiliaryoptics 540 may be moved using an auxiliary actuator (not shown) which iscontrolled by the controller 530.

The optical imaging system 500 includes a camera 535 (or video-scope)that is configured to capture image data from the optical assembly 505.Operation of the camera may be controlled by the controller 530. Thecamera 535 may also output data to an external system, such as anexternal workstation or external output device, to view the capturedimage data. In some embodiments, the camera 535 outputs data to thecontroller 530, which, in turn, transmits the data to an external systemfor viewing. The captured images may be viewable on a larger display andmay be displayed together with other information relevant to a medicalprocedure, e.g. a wide-field view of the surgical site, navigationmarkers, 3D images. etc. Image data captured by the camera 535 may bedisplayed on a display together with a wide-field view of the surgicalsite, for example, in a multiple-view user interface. The portion of thesurgical site that is captured by the camera 535 may be visuallyindicated in the wide-field view of the surgical site.

The optical imaging system 500 may include a three-dimensional (3-D)scanner 545 or 3-D camera for obtaining 3-D information of a viewingtarget. 3-D Information from the 3-D scanner 545 may be captured by thecamera 535, or captured by the 3D scanner 545 itself. Operation of the3-D scanner 545 may be controlled by the controller 530. and the 3-Dscanner 545 may transmit data to the controller 530. 3-D informationfrom the 3-D scanner 545 may be used to generate a 3-D image of aviewing target (e.g. a 3-D image of a target tumor to be re-sected). 3-Dinformation may also be useful in an augmented reality (AR) displayprovided by an external system. For example, an AR display may, usinginformation from a navigation system to register 3-D information withoptical images, overlay a 3-D image of a target specimen on a real-timeoptical image captured by the camera 535.

The controller 530 is coupled to a memory 550. The memory 550 may beinternal or external in relation to the optical imaging system 500. Datareceived by the controller 530 (e.g. image data from the camera 535, 3-Ddata from the 3D scanner, etc.) may be stored in the memory 550. Thememory 550 may also contain instructions to enable the controller tooperate the zoom actuator 520 and the focus actuator 525. For example,the memory 550 may store instructions to enable the controller 530 toperform auto-focusing. The optical imaging system 500 may communicatewith an external system. such as a navigation system or a workstation,via wired or wireless communication. In some embodiments, the opticalimaging system 500 may include a wireless transceiver (not shown) toenable wireless communication. In some embodiments, the optical imagingsystem 500 includes a power source (e.g. a battery) or a connector to apower source, such as an AC adaptor. The optical imaging system 500 mayreceive power via a connection to an external system, such as anexternal workstation or processor.

In some embodiments. the optical assembly 505. zoom actuator 520, focusactuator 525, and camera 535 may all be housed within a single housing(not shown) of the optical imaging system. The controller 530. memory550. 3D scanner 545, wireless transceiver, and/or power source may alsobe housed within the housing. The optical imaging system 500 may alsoprovide mechanisms to enable manual adjusting of the zoom optics 510and/or focus optics 515. Such manual adjusting may be enabled inaddition to motorized adjusting of zoom and focus.

The optical imaging system 500 may be mounted on a movable supportstructure, such as a positioning system (e.g. a robotic arm) of anavigation system, a manually operated support arm, a ceiling mountedsupport, a movable frame, or other such support structure. The opticalimaging system 500 is removably mounted on the movable supportstructure. In some embodiments, the optical imaging system 500 mayinclude a support connector. e.g. a mechanical coupling, to enable theoptical imaging system 500 to be quickly and easily mounted ordismounted from the support structure. The support connector on theoptical imaging system 500 may be suitable for connecting with a typicalcomplementary connector on the support structure, e.g. as designed fortypical end effectors. In some embodiments, the optical imaging system500 may be mounted to the support structure together with other endeffectors, or may be mounted to the support structure via another endeffector.

When mounted, the optical imaging system 500 may be at a known fixedposition and orientation relative to the support structure, bycalibrating the position and orientation of the optical imaging system500 after mounting. By determining the position and orientation of thesupport structure. e.g., using a navigation system or by tracking themovement of the support structure from a known starting point, theposition and orientation of the optical imaging system 500 may also bedetermined. In some embodiments, the optical imaging system 500 mayinclude a manual release button that, when actuated, enables the opticalimaging system 500 to be manually positioned.

In some embodiments, where the optical imaging system 500 is intended tobe used in a navigation system environment, the optical imaging system500 may include an array of trackable markers, which is mounted on aframe on the optical imaging system 500 to enable the navigation systemto track the position and orientation of the optical imaging system 500.Alternatively, or additionally, the movable support structure. such as apositioning system of the navigation system, on which the opticalimaging system 500 is mounted, may be tracked by the navigation system.The position and orientation of the optical imaging system 500 may bedetermined by using the known position and orientation of the opticalimaging system 500 relative to the movable support structure.

The position and orientation of the optical imaging system 500 relativeto a viewing target may be determined by a processor external to theoptical imaging system 500, such as a processor of the navigationsystem. Information about the position and orientation of the opticalimaging system 500 may be used, together with a robotic positioningsystem, to maintain alignment of the optical imaging system 500 with theviewing target throughout the medical procedure.

The navigation system tracks the position and orientation of thepositioning system and/or the optical imaging system 500, eithercollectively or independently. The navigation system may determine thedesired joint positions for the positioning system so as to maneuver theoptical imaging system 500 to an appropriate position and orientation tomaintain alignment with the viewing target. For example, the positioningsystem may be configured to align the longitudinal axes of the opticalimaging system 500 and the access port. This alignment may be maintainedthroughout the medical procedure automatically, without requiringexplicit control input. In some embodiments, the operator may be able tomanually move the positioning system and/or the optical imaging system500. During such manual movement, the navigation system may continue totrack the position and orientation of the positioning system and/or theoptical imaging system 500. After completion of manual movement, thenavigation system may reposition and reorient the positioning system andthe optical imaging system 500 to regain alignment with the access port.

The controller 530 may use information about the position andorientation of the optical imaging system 500 to perform auto-focusing.In particular, the controller 530 may be configured to performauto-focusing operations for a camera of the optical imaging system 500.For example, the controller 530 may determine a working distance betweenthe optical imaging system 500 and a viewing target, and based on theworking distance, determine the desired positioning of the focus optics515 to obtain a focused image. The position of the viewing target may,for example, be determined by a navigation system. The working distancemay be determined by the controller 530 using information about theposition and orientation of the optical imaging system 500 and/or thepositioning system relative to the viewing target. In some embodiments,the working distance may be determined by the controller 530 using aninfrared light (not shown) mounted on or near a distal end of theoptical imaging system 500.

In some embodiments, the controller 530 may perform auto-focusingwithout reference to information about the position and orientation ofthe optical imaging system 500. For example, the controller 530 mayadjust the focus actuator 525 to move the focus optics 515 into a rangeof focus positions and control the camera 535 to capture image data ateach focus position. The controller 530 may then perform imageprocessing on the captured images to determine which focus position hasthe sharpest image and determine that this focus position is the desiredposition of the focus optics 515. The controller 530 may then controlthe focus actuator 525 to move the focus optics 515 to the desiredposition. Other auto-focus routines, such as those suitable for handheldcameras. may be implemented by the controller 530 as appropriate.

A viewing target may be dynamically defined by an operator, for example,via touch input selecting a desired target on a touch-sensitive display.by using eye- or head-tracking to detect a point at which the operator'sgaze is focused. and/or by voice command. The optical imaging system 500may perform auto-focusing to dynamically focus the image on the definedviewing target, thereby enabling the operator to focus an image ondifferent points within a field of view, without changing the field ofview and without having to manually adjust the focus of the opticalimaging system 500.

In at least some embodiments. the optical imaging system 500 may beconfigured to perform auto-focusing relative to a tracked tool, such asa medical instrument, that is used in a medical procedure. For example,the position and orientation of a medical instrument (e.g. a trackedpointer tool) may be determined, and the controller 530 may performauto-focusing to focus the captured image on a point defined relative tothe medical instrument. As the tracked tool is moved, the workingdistance between the optical imaging system 500 and a defined focuspoint of the tracked tool may change. The auto-focusing is performed ina manner similar to that as above described; however, instead ofauto-focusing on a viewing target in the surgical field, the opticalimaging system 500 focuses on a focus point that is defined relative tothe tracked tool. The tracked tool may be used in the surgical field toguide the optical imaging system 500 to auto-focus on different pointsin the surgical field, enabling a surgeon to change the focus within afield of view, e.g. focus on a point other than at the center of thefield of view, without changing the field of view and without needing tomanually adjust the focus of the optical imaging system 500. Where thefield of view includes objects at different depths. the surgeon may usethe tracked tool, e.g. a pointer, to indicate to the optical imagingsystem 500 the object and/or depth desired for auto-focusing.

The controller 530 may receive information about the position andorientation of a medical instrument. This position and orientationinformation may be received, for example, from an external source, suchas an external tracking system for tracking the medical instrument. orfrom another component of the optical imaging system 500, e.g. aninfrared sensor or a machine vision component of the optical imagingsystem 500. The controller 530 may determine a focus point relative tothe position and orientation of the medical instrument. The focus pointmay be predefined for a given medical instrument, e.g. the distal tip ofa pointer, the distal end of a catheter, the distal end of an accessport. the distal end of a soft tissue re-sector, the distal end of asuction, the target of a laser, or the distal tip of a scalpel), and isdifferent for different medical instruments. The controller 530 may usethis information, together with information about the known position andorientation of the optical imaging system 500 in order to determine thedesired position of the focus optics 515 to achieve an image focused onthe focus point defined relative to the medical instrument.

Where the optical imaging system 500 is used with a navigation system(such as the medical navigation system 205). the position andorientation of a medical instrument, e.g. tracked pointer tool 222,tracked port 210, etc. may be tracked and determined by the navigationsystem. The controller 530 of the optical imaging system 500 mayautomatically focus the optical imaging system 500 to a predeterminedpoint relative to the tracked medical instrument. For example, theoptical imaging system 500 may auto-focus on the tip of a trackedpointer tool or on the distal end of the access port 210.

FIG. 4B illustrates an example embodiment of an optical imaging system500. Specifically. FIG. 4B is a perspective view of an end-effectorwhich may house the components of the optical imaging system 500. Theend-effector may be tracked, for example, using an external trackingsystem. As shown in FIG. 4B, one or more tracking markers 1200 (e.g.spheres) may be coupled to the end-effector to facilitate tracking by atracking system. The tracking system may include, at least, a stationarytracking camera 213 which detects the positions of the tracking markers1200, enabling the tracking system to determine the position andorientation of the end-effector. The camera 535 of the optical imagingsystem 500 may be in a fixed position and/or orientation relative to theend-effector's tracking markers 1200. Accordingly, the position andorientation of the camera 535 may be tracked using the external trackingsystem. In particular. a distance between the camera 535 and a viewingtarget, such as a tracked medical instrument, may be obtained using thetracking system. That is, by tracking the position and/or orientation ofa tracked medical instrument and the end-effector using trackingmarkers, an auto-focus driven focus distance of a camera of the opticalimaging system 500 may be determined.

In at least some embodiments, the optical imaging system 500 may performauto-focusing relative to a medical instrument only when the focus pointrelative to the medical instrument is determined to be within the fieldof view of the optical imaging system 500. Where the optical imagingsystem 500 is mounted on a movable support system. such as a roboticarm, if the focus point of the medical instrument is outside of thecurrent field of view of the optical imaging system 500, the movablesupport system may position and orient the optical imaging system 500 tobring the focus point of the medical instrument within the field of viewof the optical imaging system 500, in response to input such as a voicecommand, activation of a foot pedal, etc.

The optical imaging system 500 may implement a time lag beforeperforming auto-focus relative to a medical instrument, in order toavoid erroneously changing focus while the focus point of the medicalinstrument is brought into, and out of, the field of view. For example,the optical imaging system 500 may be configured to auto-focus on afocus point of a tracked medical instrument only after the focus pointhas been substantially stationary for a predetermined length of time.e.g. approximately 0.5 second to 1 second. In some embodiments, theoptical imaging system 500 may also be configured to perform zoomingwith the focus point as the zoom center. For example, while a focuspoint is in the field of view, or after auto-focusing on a certain pointin the field of view, the user may provide command input to instruct theoptical imaging system 500 to zoom in on the focus point. The controller530 may then position the zoom optics 520 accordingly to zoom in on thefocus point. Where appropriate, the positioning system may automaticallyreposition the optical imaging system 500 as needed to center the zoomedin view on the focus point.

In some embodiments, the optical imaging system 500 may automaticallychange between different auto-focus modes. For example, if the currentfield of view does not include any focus point defined by a medicalinstrument, the controller 530 may perform auto-focus based on presetcriteria, e.g. to obtain the sharpest image or to focus on the center ofthe field of view. When a focus point defined by a medical instrument isbrought into the field of view, the controller 530 may automaticallyswitch mode to auto-focus on the focus point. In some embodiments, theoptical imaging system 500 may change between different auto-focus modesin response to user input. In particular. a user may trigger auto-focus(e.g. single or continuous auto-focus) in the optical imaging system 500via, for example, user command on a user interface. voice input, oractivation of a foot pedal (e.g. button press, acknowledgement. etc.).In various examples of auto-focusing, whether or not relative to amedical instrument, the optical imaging system 500 may be configured tomaintain the focus as the zoom is adjusted.

The optical imaging system 500 may additionally generate a depth map(not shown). This is performed by capturing images of the same field ofview, wherein the optical imaging system 500 focuses on points at aplurality of different depths to simulate 3-D depth perception. Forexample, the optical imaging system 500 may perform auto-focusingthrough a predefined depth range. e.g. through a depth of approximately1 centimeter, and capturing focused images at a plurality of differentdepths through a depth range. The plurality of images captured at thecorresponding different depths may be transmitted to an external system,such as an image viewing workstation, and the plurality of images may beaggregated into a set of depth images to form a depth map for the samefield of view.

The presently disclosed methods for performing auto-focus operations aredescribed with reference to FIGS. 6 and 7. More specifically, themethods 600 and 700 illustrated in FIGS. 6 and 7, respectively, mayenable recovery of focus in a continuous auto-focus mode for an opticalimaging system. In a medical procedure, an optical imaging system (i.e.a camera of the optical imaging system) may automatically focus on afocus point defined relative to a tracked medical instrument, such as apointer tool. When the tracked medical instrument is removed from afield of view of the camera, out-of-focus images may be produced by thecamera. In particular, a focus distance of the camera may be set torandom values as the tracked medical instrument is moved out of thefield of view of the camera.

This scenario is illustrated in FIG. 8A. The surgeon 201 may use atracked medical instrument 802 during a procedure, and the camera 535may be configured to auto-focus on a defined focus point relative to thetracked medical instrument 802. By virtue of the auto-focusing, thesurgeon 201 may be able to observe a surgical site of interest when thetracked medical instrument 802 is moved over, or in proximity to, thesurgical site. However, if the surgeon 201 removes the tracked medicalinstrument 802 from a field of view of the camera 535, the camera 535may produce an out-of-focus image. This loss of focus may, at leasttemporarily, impede the surgeon 201 from observing the surgical sitewith the desired zoom and focus using the camera 535.

A desired auto-focus scenario is illustrated in FIG. 8B. In FIG. 8B, asthe surgeon 201 removes the tracked medical instrument 802 from thefield of view of the camera 535, the focus of the camera 535 may changeuntil the tracked medical instrument is completely outside of the fieldof view of the camera 535. The focus distance of the camera 535 can thenbe returned to a most recent focus distance value that was set withintent, for example, by the surgeon 201. In particular, the focusdistance of the camera 535 may, after a period of fluctuation duringremoval of the tracked medical instrument from the field of view of thecamera 535, be set to the most recent value of focus distance that wasdetermined to be stable prior to the removal from the field of view ofthe camera 535.

FIG. 6 shows, in flowchart form, an example method 600 for recoveringauto-focus in a camera of an optical imaging system. The method 600 maybe implemented in a digital microscope system. For example, the method600 may be implemented by a controller of an optical imaging systemintegrated into a digital microscope, or similar processing unit forcontrolling operations of a camera of an optical imaging system. Theoptical imaging system is configured for continuous auto-focusing. Inparticular, the camera is configured to automatically focus to a definedfocus point relative to a tracked tool, such as a medical instrument.

In operation 602, the controller receives a signal from a trackingsystem for tracking a position of a medical instrument. The signalrepresents an indication of, at least, a position of the medicalinstrument relative to the camera of the optical imaging system. Inparticular, the signal may include a representation of the position of adefined point of the medical instrument with respect to a field of viewof the camera. The tracking system may be configured to track a point(e.g. a distal tip) defined relative to the medical instrument, and thesignal may indicate the position of this defined point with respect tothe field of view of the camera. In some embodiments, the signal mayadditionally represent an orientation of the medical instrument relativeto the camera. For example, the signal may include a representation ofan orientation of a defined portion of the medical instrument withrespect to a field of view of the camera.

In operation 604, the controller determines, based on the receivedsignal, that the medical instrument is removed from a field of view ofthe camera. For example, the controller may determine that a definedpoint on the medical instrument is positioned outside of the field ofview of the camera. In some embodiments, the controller may detect anapproximate speed of the medical instrument and determine whether themedical instrument is being removed from the field of view of the camerabased on the speed data. For example, the tracking system may beconfigured to detect an approximate speed of a defined point on themedical instrument and transmit the speed data to the controller atcertain intervals. Based on the received speed data, if the approximatespeed of (a defined point on) the medical instrument exceeds apredefined threshold speed, the controller may determine that themedical instrument is being removed from the field of view of thecamera. Alternatively, the speed data may be obtained via a finitedifference computation, as will be described in greater detail below.

If the auto-focus mode is enabled for the camera, the controller may,upon determining that the medical instrument is removed from the fieldof view of the camera, adjust a focus distance of the camera. Thisbehavior may be desirable in the continuous auto-focus mode in order toensure that the latest stable focus position is recovered and the focuscontinues to track to the surgical site. Thus, in operation 606, thecontroller retrieves, from a database, a first focus distance valuerepresenting a focus distance that was most recently set with intent forthe camera. The database may, in some embodiments, be a rolling buffercontaining recent focus distance data for the camera. For example, therolling buffer may store a predetermined number of most recentauto-focus driven focus distance values for the camera and timestampsassociated with the detected focus distance values. The focus distancevalues may be obtained, for example, from the camera or determined bythe controller itself.

The retrieved first focus distance may be a value of focus distancestored in the database with the most recent timestamp. The focusdistance values that are stored in the database may represent, forexample, stable focus distances, or focus distances corresponding toin-focus imaging by the camera. The stable focus distances may, in someembodiments, be associated with dwell periods which last longer than apredefined threshold. A dwell period refers to a period of time duringwhich a tracked medical instrument “dwells” or remains in relativelyfixed distance with respect to the camera. A longer dwell period mayrepresent steady or stable positioning of the medical instrumentrelative to the camera, whereas a short dwell period may correlate tomovement of the medical instrument. The controller (or the cameraitself) may store those focus distance values that are determined to bestable based, for example, on dwell periods.

In operation 608, the controller automatically updates a focus distanceof the camera to the retrieved first focus distance value. That is, thefocus distance of the camera may be automatically set to the first focusdistance value determined by the controller, and not a different focusdistance value which may result from the default auto-focus behavior forthe tracked medical instrument.

FIG. 7 shows, in flowchart form, another example method 700 forrecovering auto-focus in a camera of an optical imaging system. As forthe method 600, the method 700 may be implemented in a digitalmicroscope system. For example, the method 700 may be implemented by acontroller of an optical imaging system integrated into a digitalmicroscope, or similar processing unit for controlling operations of acamera of an optical imaging system. The optical imaging system isconfigured for continuous auto-focusing. In particular, the camera isconfigured to automatically focus to a defined focus point relative to atracked tool, such as a medical instrument.

The method 700 employs a speed-based approach to determining a stablefocus distance value when recovering continuous auto-focusing for acamera. In particular, the method is based on an assumption that a focusdistance that is set with intent by an operator of the camera will havean associated dwell period during which a tracked tool driven focusdistance has an associated speed that is consistently below a predefinedthreshold. That is, a stable focus distance value may correspond to aperiod of time during which a tracked tool is moved at a speed that isless than a threshold. The continuous auto-focus of the camera may berecovered by automatically setting a focus distance of the camera to themost recent such stable focus distance value.

In operation 702, the controller stores focus distance values andassociated timestamps for the camera in a database. The database may,for example, be a rolling buffer for storing a predetermined number ofmost recent focus distance values for the camera.

The controller detects, in operation 704, that a tracked medicalinstrument is out of field of view of the camera. For example, thecontroller may receive, from a tracking system for tracking the medicalinstrument, a signal representing an indication of the position and/ororientation of the medical instrument relative to the camera. Based onthe received signal, the controller may determine whether the medicalinstrument is out of, or being removed from, the field of view of thecamera. For example, the controller may compare coordinates associatedwith the camera's field of view with coordinates of the medicalinstrument (or a defined point relative to the medical instrument) inthe same coordinate space. Upon comparing the coordinates, thecontroller may be able to determine whether the medical instrument iswithin, or falls outside of, the bounds of the camera's field of view.

In operation 706, the controller obtains, for each stored focus distancevalue, an associated speed value. The speed values represent theinstantaneous speeds of the tracked medical instrument at the respectivefocus distances. That is, the associated speed value represents anapproximate speed of a tracked point of the medical instrument at thefocus distance. In at least some embodiments, the associated speed for afocus distance value may be approximated numerically. For example, theassociated speed may be obtained based on a finite differencecomputation using stored focus distance values for the camera. That is,the rates at which the focus distance values change, presumably as aresult of movement of the tracked medical instrument, may beapproximated. Various techniques for computing finite differenceapproximations may be suitable. In some embodiments, the calculation ofweights in finite difference formulas during the calculation ofapproximate speeds may be based on an approach proposed in “Calculationof Weights in Finite Difference Formulas” (Fornberg, SIAM Rev. vol, 40,no. 3, pp. 685-691, September 1998), which is incorporated herein byreference. The controller may additionally, or alternatively, beconfigured to compute a speed curve associated with the stored focusdistance values. In particular, a speed curve representing approximatespeeds of the medical instrument which are associated with the storedfocus distance values may be computed. Such speed curve may facilitateidentification of focus distance values that are “stable”, or associatedwith dwell periods during which the associated speed falls below a giventhreshold.

The speed values that are approximated numerically may be stored inassociation with the respective focus distance values. In operation 708,the controller retrieves a first focus distance value that is associatedwith a speed which is below a predefined threshold speed. In particular,the controller retrieves the most recent such focus distance value. Insome embodiments, the predefined threshold speed may be 0.1 meter persecond. The controller may retrieve from a data buffer the focusdistance value with the most recent timestamp and which is associatedwith a speed that falls below the predefined threshold.

Upon retrieving the first focus distance value, the controllerautomatically updates a focus distance of the camera to the retrievedvalue, in operation 710. That is, the focus distance of the camera maybe automatically set to the first focus distance value in order torecover continuous auto-focus and for the focus to track to the surgicalsite.

The various embodiments presented above are merely examples and are inno way meant to limit the scope of this application. Variations of theinnovations described herein will be apparent to persons of ordinaryskill in the art, such variations being within the intended scope of thepresent application. In particular, features from one or more of theabove-described example embodiments may be selected to createalternative example embodiments including a sub-combination of featureswhich may not be explicitly described above. In addition, features fromone or more of the above-described example embodiments may be selectedand combined to create alternative example embodiments including acombination of features which may not be explicitly described above.Features suitable for such combinations and sub-combinations would bereadily apparent to persons skilled in the art upon review of thepresent application as a whole. The subject matter described herein andin the recited claims intends to cover and embrace all suitable changesin technology.

1. A processor-implemented method for performing auto-focus in a camera,the method comprising: receiving, from a tracking system for tracking aposition of a medical instrument, a signal; determining, based on thereceived signal, that the medical instrument is removed from a field ofview of the camera; in response to determining that a continuousauto-focus mode for the camera is enabled: retrieving, from a database,a first focus distance value representing a focus distance that was mostrecently set with intent for the camera; and automatically updating afocus distance of the camera to the first focus distance value.
 2. Themethod of claim 1, further comprising storing, in the database, apredetermined number of most recent focus distance values for the cameraand timestamps associated with the focus distance values.
 3. The methodof claim 2, wherein retrieving the first focus distance value comprises:obtaining, for each focus distance value stored in the database, anassociated speed value, the associated speed value representing anapproximate speed of a tracked point of the medical instrument at thefocus distance; retrieving, from the database, a most recently storedfocus distance value having an associated speed that is below apredefined threshold speed.
 4. The method of claim 3, wherein theassociated speed is obtained based on a finite difference computationusing focus distance values stored in the database.
 5. The method ofclaim 3, wherein the predefined threshold speed is 0.1 meter per second.6. The method of claim 1, wherein the camera is configured toautomatically focus to a predetermined point relative to the medicalinstrument.
 7. The method of claim 1, wherein the continuous auto-focusmode is activated via a voice input or activation of a foot pedal. 8.The method of claim 6, wherein detecting that the medical instrument isremoved from the field of view of the camera comprises determining thatthe predetermined point is no longer within the field of view of thecamera.
 9. The method of claim 1, wherein detecting that the medicalinstrument is removed from the field of view of the camera comprisesdetermining that an approximate speed of the medical instrument exceedsa predefined threshold speed.
 10. The method of claim 2, furthercomprising computing a speed curve representing approximate speeds ofthe medical instrument which are associated with the stored focusdistance values.
 11. A navigation system to support a medical procedure,the navigation system comprising: a tracking system for tracking aposition of a medical instrument; a surgical camera for imaging a targetsurgical site; and a processor coupled to the tracking system and thesurgical camera, the processor being configured to: determine, based ona signal from the tracking system, that the medical instrument isremoved from a field of view of the surgical camera; in response todetermining that a continuous auto-focus mode for the surgical camera isenabled: retrieve, from a database, a first focus distance valuerepresenting a focus distance that was most recently set with intent forthe surgical camera; and automatically update a focus distance of thesurgical camera to the first focus distance value.
 12. The navigationsystem of claim 11, wherein the processor is further configured tostore, in the database, a predetermined number of most recent focusdistance values for the surgical camera and timestamps associated withthe focus distance values.
 13. The navigation system of claim 12,wherein retrieving the first focus distance value comprises: obtaining,for each focus distance value stored in the database, an associatedspeed value, the associated speed value representing an approximatespeed of a tracked point of the medical instrument at the focusdistance; retrieving, from the database, a most recently stored focusdistance value having an associated speed that is below a predefinedthreshold speed.
 14. The navigation system of claim 13, wherein theassociated speed is obtained based on a finite difference computationusing focus distance values stored in the database.
 15. The navigationsystem of claim 13, wherein the predefined threshold speed is 0.1 meterper second.
 16. The navigation system of claim 11, wherein the surgicalcamera is configured to automatically focus to a predetermined pointrelative to the medical instrument.
 17. The navigation system of claim11, wherein the continuous auto-focus mode is activated via a voiceinput or activation of a foot pedal.
 18. The navigation system of claim16, wherein detecting that the medical instrument is removed from thefield of view of the surgical camera comprises determining that thepredetermined point is no longer within the field of view of thesurgical camera.
 19. The navigation system of claim 11, whereindetecting that the medical instrument is removed from the field of viewof the surgical camera comprises determining that an approximate speedof the medical instrument exceeds a predefined threshold speed.
 20. Thenavigation system of claim 11, wherein the processor is configured tocompute a speed curve representing approximate speeds of the medicalinstrument which are associated with the stored focus distance values.